Colearning Project - CS 290 C SP 2007

Overview Colearning in the expert setting is explored by examining the problem setting and running experiments. Implementations of a variety of existing algorithms from the static and shifting experts framework of Bousquet and Warmuth [2], an adaptation developed by Abernethy, Bartlett and Rakhlin [6] designed to mix weights over tasks, and an ad-hoc task frequency based mixing algorithm are used. A variety of test cases ranging through the sequential to shifting task problem space serve as experimental input for these algorithms. Caruna [3] describes multitask learning in terms of combining inputs from multiple tasks as inputs to the hidden layers of a single neural network. Bousquet and Warmuth [2] describe the " shifting experts " setting where instances are related over a local, but otherwise unknown timeframe, refered to as a " segment ". A segment loosely corresponds to the idea of a task, but in the shifting experts setting, responding to unknown segment boundaries is part of the work the algorithm must do to minimize regret. In the sequential multi-task problem, the correspondence between multi-task colearning and shifting-experts is most compeling, and it is for this version of the problem that Abernethy, Bartlett and Rakhlin [6], adapt the shifting-experts framework to a " task mixing " algorithm that has proveably good bounds. This is described in more detail in section .